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ABSTRACT 

Preoccupation with the science of educational 
evaluation has led educators to overlooJe the art of evaluation, the 
ability to make quick and accurate assessments, usually without 
statistical props. This paper suggests common^sense steps to insure 
accurate assessment of innovative structures* (1) Know the nature and 
purpose of innovation* An urban school system may innovate in 
desperation; suburban schools may make administrative or procedural 
changes out of jealousy of another suburb* (2) Knoir how interference 
from the physical product affects sound assessment. The 
attractiveness of a set of books or other materials may blur an 
educator's judgment. (3) Know how interference from human factors 
affects assessment; nice teachers or appealing children can distort 
analysis of real issues. (4) The relationship between claims and 
reality must be carefully assessed. The innovation's purpose must be 
identified and its performance of that goal evaluated. (5) Know the 
theory or knowledge base underlying the structure or program. It must 
be determined whether the program successfully fits its theoretical 
teaching and learning framework. (6) Know which areas are susceptible 
to innovation and which are resistant. The public will often resist 
innovation in certain areas and attempts at change will fail* (CHK) 
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The problem of how to evaluate or assess educational irnovations has 
alvays beftn difficult, A quick search of books and articles on educational 
evaluation will yield a number of works which are heavily laden with terms 
such as feedback factor , ipsative analysis and process determinant s as well 
as a bewildering array of statistical displays. It appears that soiaehcv the 
art of assessing has been turned into the science of evaluation, and this 
magic has been wrought by the mystery of mathematics. 

There is an increasingly old proverb in education in which it is main* 
taincd that educators use evaluation the way a drtmk uses a lamp-post: more 
for support than for enlightenment. To this observation it has become 
increasingly clear that evaluation is also being used to impress with our 
scientific know-how. If statistics can be applied, somehow we feel better 
about it all. 

In light of this situation, I would like to suggest that soma v^ry 
important ingredients are being overlooked. However important the s cience 
of educational evaluation might be, the practitioners still have to depend 
on the art of evaluation - that ability to make quick and accurate assess* 
monts of materials, programs and staff, usually without the benefit of chl- 
squares or regression analyses. Like effective management techniques, good 
assessment follows the principles of simple common sense. Oddly enough, 
when faced with a new technique or product, many administrators forsake the 
principles of common sense and are taken in by the pressrires of the situation. 
There are several safeguards against such failure and it is the purpose cf 
this paper to suggest six positive steps to be taken to Insure accurate assess- 
ment of innovative structures and programs in the educational setting. 



1. The natu r e of innovation: Know ^.7hv vou are innovating 

For one thing, the nature of educational innovation should be clearly 
understood. There are two common motivations for a school system to innovcce 
at all. One motivation can be called the Durban" model of innovation. We 
call it "urban" because it describes the major locus of this type of 
innovation. Philadelphia adopted the Pa.ckway Plan, a unique solution to 
classroom crowding in the Philadelphia schools when the school system^ s back 
was against the wall. When things are so dismal that there is nothing else 
to do, a school district is sometimes driven to exciting innovative practice. 
Desperation of this sort is sometimes the mother of creativity but it can 
also be the first step in a move toward futility. An example of this sort 
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of educational practice is the now discredited but formerly acclaloed practice 
of parfomonce contracts. Performance contracting in reading, at any rate, 
proved to be a dismal failure for a number of reasons but primarily bacause 
more attention was given the method of finding accountability than was given 
the substance being measured. In any case, the urban motivation for 
innovating stems from a kind of desperate effort after all else has failed* 
Like prayer, it is considered vjhen there is no where else to turn. The 
Urban innovation model grows out of a definite sense of need. 

In contrast to the urban model is the "suburban" motivation for 
innovating in the schools. Suburban schools may have genuine problems but 
they are much less frequently recognized. Thus, when innovation is con* 
sidered in a suburban school, it is done so more out of a desire to keep up 
with another suburb than it is from a recognized failure • This model is 
sometimes referred to as the jealousy model in educational innovation. It 
produces less dramatic innovations (the development of a middle school, for 
example) and almost always these innovations are administrative or pro- 
cedural rather than content based. New arrangements of schedules, classroom 
management or building design doisinate in the suburban model of educational 
innovation. 

To be sure, there are other reasons for innovating besides desperation 
and jealousy, but examination of actual practice has clearly indicated that 
these two motivations are most commonly used. One might innovate, for 
example, because new knowledge is discovered or because internal evaluation 
has revealed a specific need but seldom does this appear to be the case In 
the public schools. A commonly used model of innovation seems to be 
illuminative here. 




Shuy 
Page 3 



whether It relates to ojedicine, agriculture or education, innovation neema 
to follow a normal bell curve. At one end are the laggards', vrtio never seem 
to make it and, it is hoped, ultimately drop out. At the opposite end are 
the innovators, those who actually will «eek out the new idea and bring it 
back and apply it. Such practitioners are unusual and they frequently do 
not last long. Their strength becomes their weakness and they are thought 
of as kooks by the non-innovators, who either grow to resent them (forcing 
them out of their field) or become afraid of them that they the Innovator 
is promoted out of the competitive arena. Early adaptors are not quite 
so creative or adventuresome* They do not go after the new idea but, if 
it should come to them, they are perfectly capable of adapting it* Most 
school systems are not innovative to the extent that they go after the new 
idea, modify it and apply it. School systems offer little time for this 
sort of thing. On the other hand, many are clearly the early adapters of 
new ideas, provided that the idea is convincingly presented. When such 
occasions arise, a number of factors can combine to interfere with a wise 
decision. Some of these factors include the interference from the physical 
product, from the people or human factors involved, from the relationship 
between claims and reality and from the theory underlying the innovation. 
Let us examine each of these potential interferences in more detail. 



2. Interference from the physical product: Know how this affects sound 
assessment s 

It seems unnecessary to have to say that the attractiveness o£ 
the product, for example a set of student books, will interfere with our 
objective judgment of this product. Yet this is a critical interference 
factor and one to which publishers are particularly sensitive. It seems 
embarrassing to admit that we like a social studies series because it is 
attractively illustrated but there is almost no way around this issue. 
Recently I participated in a two day site visit and evaluation of a 
government sponsored program to develop reading materials for the middle 
grades. As is often the case, the student materials were sent to the 
evaluators well in advance of the site visit. The stories were clever and 
the evaluation experience was pleasant, for it is difficult to become very 
negative about children's literature. I found myself more and more con- 
tented with the project simply because of the glow created by. the happy 
stories. Naturally we want the student materials to be attractive and to 
provide the same sort of happy affecc on the children who read them, but 
in our role of assessors of materials and pro^^rams we cannot let the aura 
of this aspect to defer us from ouv analytic role. Educators are par- 
ticularly susceptible to this danger simply because we deal in children 
and with products developed specifically for children. 
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3 . Interfe rence from htx m an factors; knew how this affects as<:e88 m ent 
of Innovations 

On many occasions I have participated in school visits, evaluations 
or ceremonial involvements which have included actual visits to real class- 
rooms. Here the dangers of subjectivity are even greater for the in- 
fluences of children on sympathetic adults can be tremendous. I have 
actually witnessed a miserably taught class using incredibly irrelevant and 
useless materials and I still went away with a warm glow for the children 
which neutralized in otherwise Justifiably negative assessment. Even in 
the program evaluation of adults who are developing teaching materials, 
the opportunity for neutralizing negative feelings is increased with the 
addition of human contact. 

In the same site visit evaluation of reading materials noted abovQ, 
I had to consciously combat my tendency to rank the program too positively 
first because of the effect which the children's level materials had on^ 
me and then because of the effect of the nice people who were developing 
the program. What the materials and the staff failed to provide was a 
picture of the rationale and dynamics of the project. To get at these, 
I gave an assignment to be completed after the site visit ended. It 
contained several questions to be answered in the period of the two weeks 
following. One question asked the project leaders to provide a rationale 
describing the uniqueness of the program (it was said to be individualized 
and based on syntax processing). A second question asked for the same sort 
of rationale only in, this case, it was to be written for the parents. 
Other questions related to different aspects of the project which are not 
relevant to this discussion. The point which I wish to make here is that 
to obtain a clear judgment of the quality of the program, it was necessary 
to consciously guard against over-reliance on those aspects of the program 
which carried heavy emotional, empathetic or nostalgic appeal and to push 
toward analysis which dealt with the specific issues to be assessed. 



4. Interference from the relationship of claims and reality: An important 
criterion for assessing innovations . 

Assessing an educational program or structure is a great deal 
like reviewing a book. The greatest failure and inaccuracy in book reviewing 
grows out of the refusal to answer three basic questions: 

1. What does it say it will do? 

2. Does it do it? 

3. Does it do it well? 

Many books are critici±ed by reviewers for not doing something, they never 
promised in the first place. When D.H. Lawrence ^s novel. Lady Chatterlv^s 
Lover > was reviewed negatively in Field and Stream for its failure to 
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accurately describe the British gameskeeper *s daily routines, one can only 
wonder what was in the reviewer's mind. The book is a novel. As such, the 
accurate portrayal of occupational roles is, at best subsidiary to the major 
goal of entertainment or revelation of some larger theme. It Is grossly 
unfair to expect a reading series to make a direct development to spelling 
skill development. It can be expected that such a series will not retard 
spelling skills and it can be hoped that reading and spelling skills will 
be mutually supportive, but if the series claims only to develop reading 
ability, we cannot fault it for not doing something it does not promise or 
intend to do. 

The critical question, however, is whether or not the product actually 
does what it says it will do. In order to assess this effectively, we must 
have a clear notion of what the task is that is being promised. On the 
surface, this may seem easy, but. things are not always what they seem. 

Nobody would agree that by teaching a person to use an automobile 
rear -view mirror we have taught this person to drive a car. For that 
matter, even learning to dribble a basketball is not an Indication that 
the learner is a good basketball player. Yet we tnake similar assessments 
about educational products, failing to realize that the gestalt is greater 
than any of its sequential, component parts. Learning to read, for example 
involves a cumulative acquisition of early skills which, once focused more 
and more on larger units of processing. Consequently, any materials which 
claim to teach reading by dealing only with an early skill in the reading 
process, such as phonics, are making clear statements about the dozens of 
other decoding skills and comprehension measures which such a program dees 
not deal with. 

An elementary program, for example, which deals with the basic 
skills should distinguish between the acquisition of these skills and the 
learning of the behavior toward which these skills procede. In fact it may 
be useful to clearly distinguish among skills, concepts and behavior, so 
that it will be easier to determine exactly what an innovative program or 
product really proposes. 

A few years ago a USOE advisory board had as its task the supporting 
and monitoring of various projects which were directed toward teaching future 
teachers certain concepts and skills which were determined to be crucial. 
The hitch was that the skills projects and the concepts projects were under 
separate funding and required different treatment. It was necessary, of 
course, for the advisory board and the projects themselves to distinguish 
between skills and concepts in this way. As easy as this sounded, we 
really had no good working definitions ^7hich suited our purposes. With 
typical government efficiency it was decided that various leaders In the 
field of education would be called together and work for two days to decide 
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this issue for once and for all. Therefore, at a Chicago airport hotel this 
group met and deliberated. It found little trouble in determining the pro- 
jects which were clearly concept oriented but it had a very difficult time 
defining the skills. After considerable deliberation, it was determined that 
a useful model might be borrowed from the field of linguistics. Linguists 
refer to the deep structure of language and to its surface structure. Rules 
are the devices which bring deep structures to the surface representations, 
as follows; 



Figure 2: 




DEEP STRUCTURE 



As Figure 2 indicates, the same aspect of deep structure may, through the 
use of different rules, lead to three different surface structure mani- 
festations. Naturally this is only a schematic representation and the 
situation can be infinitely complex. Borrowing this phenomenon as a 
metaphor, the advisory panel decided that concepts were like deep structure, 
that behavior is like surface structure and that skills are like rules, as 
figure 3 indicates: 

Figure 3: 

BEHAVIOR 




COIICSPTS 
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Such a model, as simple as it appears, provides, access to complex explanations. 
Like grammatical rules, skills are relatively meaningless unless they are 
related to the concept from which they emanate and to the behavior toward 
which they are leading. A skill, by itself, is an empty thing. It is not the 
skill in and of itself that we wish to develop, it is the larger gestalt 
which embraces the concept, the skill and the ultimate behavior. Skills 
are very difficult to define, in fact, unless their definition gDows oiit 
of such a gestalt. 

Thus learning to drive a car includes a- number of skills which 
derive from important concepts and lead toward relevant behaviors. Learning 
how to use a rear-view mirror is one such skill but to study "rear^view 
mirror looking" as an end product is a relatively futile exercise. The 
Implication of this principle for assessing educational innovations should 
be rather clear. Many innovative products and strategies present only a 
part, often a very small part, of the desired ultimate behavior. 

One of the major failures of innovative products in education, in 
fact, is that one segment of the concept-skill-behavior process is singled 
out with the claim that by improving that segment the entire process will 
be enhanced. This is not to deny that the innovation will improve that 
particular pieces of the education process. Perhaps it will help. What 
I am warning about is, on the other hand, that helping that part of the 
process in no way assures that the learning gestalt will be similarly aided» 

Evidence to support this warning can be drawn from everyone's cotnmon 
experiences. One of my own will probably suffice. Not long ago my older 
son took drivers training in high school.. For the duration of his course 
and for some time thereafter he was very critical of my driving. It appears 
that I violate many of the skills required of beginoing drivers. I do not 
look at the rear view mirror often enough. I do not position my hands on 
the steering wheel properly. I do. not do many other things exactly as he is 
being taught. I do not doubt the usefulness of his learning these early 
skills but I \iuestion the usefulness of maintaining a conscious awareness of 
these skills for very long into the learning process. As soon as possible, 
these skills should be submerged into some sort of natural behavior in which 
it is no longer necessary to think about them. Consider what would happen 
if you should return to the beginner level of skill development related to 
walking and consciously put forth each skill involved in moving your legs 
in walking. Chances are that you would fall. The same dangers exist in 
driving and even In talking. On one occasion a few years ago Charlie Brown, 
the round headed character in the ccmic strip. Peanuts, became "aware of his 
tongue." This awareness prevented him from talking for three or four days. 

A major question to be asked of any innovative materials or program, 
then, is whether the promise relates to reality. Does the innovations itself 

8 
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have the ability to do what is claimed for it? To answer this question, 
the buyer will have to know what is involved in the larger process, the 
gestalt, in order to determine whether or not the component process re- 
presented by the innovation can or does realistically represent the claim. 

To my thinking, this ability to distinguish the component part from 
the whole and to weigh them in relationship to each other is the most 
difficult but highly important task in assessing the innovation. Other tasks 
are somewhat easier to deal with but by no means are they effectively carried 
out by school districts in such assessment. 

One such task is to determine whether or not the gain factor claim 
is realistic. Can children learn to read in five weeks? Is it reasonable 
to assume that by feeling balloon shaped alphabet letters, the child will 
speed his reading? Is it realistic to think that minority children will 
learn to read only after they are taught to speak Standard English? These, 
and many other questions, can be asked of almost any educational innovation. 
We still do not have adequate research proof which tells us that bussing 
helps children learn better* Knowledge of the research base upon which 
educational innovations rest is a critical factor here. Innovations should 
make such knowledge available and understandable. If the classroom 
architecture which advocates no walls is said to be efficient, ask its pro- 
ponent to explain why and upon V7hat evidence his reasons rest. Once on an 
HEW site visit to a large Southern city I asked the Superintendent of Schools 
why that city was architecturally committed to building all new buildings 
on the ''no walls" basis. His response indicated ignorance beyond belief. 
He said that the ''open classroom" concept was proven highly successful and 
that this motivated their decision. Several things were wrong with his 
answer. For one thing schools without walls are not the same thing as the 
open classroom. Even the legitimate definitions of open classroom leave 
a great deil of confusion. I had never run into such a humorous misunder- 
standing of the term before nor have r since. The really sad part of this 
event, however, was that even after taking me on an inspection of one of 
his model "open classrooms" he could not give me one good reason for such 
an architectural commitment. Obviously, he had not even considered the 
claims of the innovation with regard to realistic gain factors. 

Another task in dTetermining the relationship between the innovation's 
claim and realistic possible outcomes is to decypher and cut through the 
buzz words. Teaching modules > for example, is a term which embraces a wide 
range of potential descriptions. Buzz words which become buzz initials are 
even more mysterious and powerful. C.3»T.E. developed out of a legitimate 
need but, like many good ideas, can be carried to unconscionable conclusions. 
Inordinate Isolated and indiscrimant trcasurement of individual competencies 
can provide a complex and Impressive display of information, but, by its 
very Isolation of items, it can also disguise reality. 
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It is truism that we should not be taken- in by buzz-words. In some 
cases, the buzz-words themselves reveal doubts about the innovation. Any 
program which offers teacher-^roof materials is so insulting to the teaching 
profession that the product can almost immediately be abandoned. Any^reading 
program that labels itself "the linguistic approach** can be guaranteed to 
not be acceptable to linguists none of whom believe that linguistics is a 
methodology or that such a thing as "the linguistic approach" actually 
exists. If the buzz words talk-down or go out of their way to overly impress, 
thl« is a clear signal of trouble within. The innovation which must be sold 
by buzz-words probably will not stand the tests of time or analysis. 



5. The Theory Underlying the Structure or Program; \ critical measurement 
point In judging and innovations . 

It may be the height of arrogance to suggest to a group of 
practitioners and administrators of practitioners that theoretical concerns 
can in any way be helpful in guarding against error in assessing educational 
innovations; but this is exactly what I am going to do. For reasons which 
are unknown and unclear, the field of education has allowed certain aspects 
of common sense to escape. Learning thoory, culture theory and management 
theory, to name only a few of the theories which can be useful to educators, 
have been singularly ignored in almost all teacher training programs, under 
the mistaken assumption that teachers can be trained to behave in the class- 
room without first understanding the framework into which such behavior fits. 
In fact, to knowledge, education is the only field which trains its 
practitioners first in methods and then brings them back later, if at all, 
for the theoretical underpinnings of these methodologies. 

The need for an overview, a framework or a theoretical base can be 
seen in most avenues of life if meaningfulness is ever achieved. The Job 
of an assembly line worker who turns only one screw is made meaningful by 
the knowledge that this screw performs a small function in^the overall 
workings of the final product. The absense of such a framework can be 
personally devestating and, in terms of classroom teaching, a learning 
nightmare . 

I once experienced what has to be the worst example possible of this 
sort of problem. The success of certain English teaching materials used 
with Mexican Americans in New Mexico beca.tic known to a school administrator 
in Mississippi who concluded that if the materials worked with the Mexican 
Americans, they would also be good for the Black children in his system. 
In an effort to check on this amazing assumption, the federal sponsor of 
this project sent a team of observers to the rural Mississippi setting to 
assess the situation. What we saw was a Black teacher trying to teach 
about ten Black children to produce the aspirated /p/ sound which occurs in 
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words like £in, in contrast to the unaspirated /p/ sound in words like spin . 
She placed the children in a circle, gave each a lighted match, then went 
around the circle asking them to say £in, an exercise in which the aspiration 
of the /p/ is supposed to blow out the match. In her haste to move quickly 
the teacher forgot the appropriate key word and substituted spin instead. 
The conclusion was pure chaos. The teacher did not understand the reason 
why she was doing what she was doing. Indeed, there really was no good 
reason since the materials were developed for children with entirely different 
needs. Native Spanish speakers do not aspirate such sounds but all native 
English speakers do. Thus, the materials were totally misapplied to this 
population. If the teacher had known whjr she was doing what she was doing, 
she might have easily determined why the key word, pin, was important and 
the contrast word, S£in, was wrong. The children, who didn't have the 
problem being treated in the first place, found the exercise interesting 
enough, at least until they could get their matches blown out by saying 
spin into the flame. Several burned their fingers. The teacher was ' / . 
embarrassed beyond belief. The teaching situation was a total disaster. 

The need for an overview, framework or theory in teaching was first 
brought home to me when I took my methodology course as an undergraduate 
preparing for a secondary certificate. For reasons which I still do not 
understand, I did ny practice teaching before I took my methodology course. 
Therefore, when various methods were discussed, I felt rather good about 
them, quite in contrast with the reactions of the other students in that 
class. Since it was not like me to appreciate a class in which other 
students were disinterested, I can only surmise that I liked the course 
because I had been there and the abstractions of methodology were meaningful 
to me because I had an experience upon which to hang them. 

It is often the obvious questions which are the least visfble in the 
development of efficient programs. Professional fund raisers, for example, 
operate on a ninnber of obvious principles which, if not observed, permit 
little opportunity of success. Recently the head of institutional develop- 
ment of a large Eastern University listed a few of these obvious principles: 

1. to get money, you must go to organizations which have money. 

2. you don't get money unless you ask for it. 

3. Seek funding when you need it. 

4. It is easier to get money for specific ideas than for general ones. 

5. You get more money for larger ideas than for smaller ones. 

Education sometimes suffers from not asking the obvious questions such as 
these. Our confusion over certain basic value issues stems from some rather 
strange misconceptions of this sort: 
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1, Being right is good and being wrong is bad, 

Z. Because something is worth learning, it's worth learning right Way. 

3. Children are inferior versions of adults. 

4. Constancy is good; variability is bad. 

5. One methodology is better than another • 

It may not be critical for administrators and practitioners to know learning 
theory in the advanced technical sense, but it jLs necessary for them to know . 
enough about learning theory to be able to reject educational innovations 
which embody hollow principles such as the above. A presentation such- as 
this does not provide the time or magnitude to illustrate the need' to under 
stand and utilize a theory of learning, a theory of language and A theory 
of culture in all subjects for all ages. Instead, let me illustrate the 
usefulness of such theoretical knowledge in one subject: reading. 

A good reading program (and any innovation which attempts ,to relate 
to reading) will not be blind to what is known about how learning takes 
place. Nor will it knuckle under to proponents of the superiority of one 
theory over another when ai* at table counter-information is available. 

A reading program or innovation which is based on the assuqiption that 
all learning is behavioral, for example, will need to account for a great 
deal of cognitive gain for which no apparent behavioral stimulus exists. 
Most reading programs make exactly this assumption, however, proce^ing as - 
though the act of reading is little more than the piling up of one early 
behavioral skill on top of another. This sort of assumption is very popular 
for a number of reasons. For one thing, it feeds on the public impression 
that basics should be the focus of all education* And what could be more 
basic than early skills? Secondly it does with the aspects of learning which 
are easiest to measure. Skills are easier to assess than are the concepts 
which the skills presumably implement. Third, skills are isolatable while 
the fact that the gestalt of learning merges, selects and reinterprets 
component skills which are involved is very messy to understand, much lesis 
to display. Consequently, we tend to see materials, including Innovative 
materials, which buildfon the popular assumptions of the public, are easily 
measurable and possible to isolate for inspection. The great fact of 
behavioral objective writing and measuring, for example grows out of such 
'^alse assumptions. 

In the case of reading, we have a rather clear example of a mixture 
of early behavioral skills, later cognitive strategies and a potential for 
cultural interpreHation and individual learning style. The following 
figure will be illustrative in this regard: 

12 
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It should be clear, however, that this schematic illustration is not 
a description based on research but rather it is a reasonable estimate of 
what is likely to be the case once the necessary research has been done. Of 
particular importance is that it displays letter-sound correspondence as 
crucial at the onset of learning to read, then decreasingly important as the 
learning to read process develops. Similar progression can be noted for 
each of the other language accesses, with particular focus, in the case of 
pragmatics, on the increasing significance of context and discourse* Note 
especially that both accesses are available and important at the onset of 
learning to read but of relatively low cruciality at that time. As the 
learner continues to progress however, he calls less and less on the T7ord 
to sub-word level accesses and more and more on the language accesses that 
are larger than word level* 

At this point it should be noted that most language learning activity 
parallels the learning to read progression insofar as the early stages of 
learning are relatively clear cut and show obvious gains whereas the middle 
level and advanced stage of language learning are loss well-known and 
obvious. That is, in almost every case, the stages in the beginning courses 
in language learning are relatively well known and measurable but, as the 
learner progresses, thb exact stages in his program are less and less clear. 
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From a conxnerclal viewpoint, we know considerably more about how to con- 
struct introductory courses than we do about how to construct advanced ones* 

The parallels to reading instruction should be clear. Historically 
we have developed reasonably good onset reading programs but increasingly 
ineffective advanced ones* Most children who are learning to read hhow 
predictable gains during the first year or so, then demonstrate, according 
to our admittedly weak measurement system, progressive fall off for the 
next few years* One reason for this fall off is that the teaching program 
continues to focus on onset skill development at stages in which more 
appropriate strategies would involve larger and larger chunking of the la 
language accesses. 

The learning theory embodied here is quite simple* It argues that 
early entry skills are rather behavioral and that such skills are not pre- 
served consciously by the learners. In fact, the preservation of awareness 
of these skills can be both boring and counter-productive* Indeed, there 
is no reason to believe that because the skill is worth learning, that is 
worth remembering after later, higher learning strategies » have replaced it* 
Educators tend to see only pieces of the learning process and to assume 
that these pieces are all equal* 

This figure displays the usefulness of early behavioral skills at 
the onset of learning but also shows that such skills decline in cruciality 
as the learner progresses to higher level cognitive processes* Likewise the 
cognitive strategies (processing at the sentence and meaning levels) are 
present but not all important at the onset of reading but that they increase 
in significance as the readers ability increases. 

It should be clearn, then, that knowledge of learning theory can be 
critical In assessing a reading program. Any innovation which stresses only 
letter-sound correspondence, only word-level processing or only sentence 
processing tends to present only a part of the gestalt of reading. Yet 
this principle is one of the least known and little used by school systems 
in selecting a reading program. Such systems are taken-in by every fad 
in the reading business, largely because they fail to hold an adequate theory 
of reading. The burden of this paper, in fact, is not that there are a 
dozen or so easy tricks or check points for administrators to use in the 
assessment of innovative structures and programs* What I am saying, quite 
tha contrary, is that to be able to assess innovations, one must have the 
knowledge base and theory to assist in making such assessments* 

Nor does this imply that all adoiinistrators have to know every thing 
about all subjects. School districts should have competent people upon 
whom reliance can be placed for supplying such knowledge and theory* In 
the case of reading, however, this may prove difficult since the field is 
one which might be considered highly susceptible to unfounded innovation* 
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Where can the administrator get this knowledge base or theoretical 
underpinning so critical for assessing innovations? The burden is on 
the producer and an effective one will provide it. This may be too much 
to ask, however, the administrator may need to do something more drastic* 
He can't be expected to know everything and it would seem that a good way 
to go would be to build a network of knowledgable people or organizations 
from whom to get spot-check assessment help. Language arts specialists 
in Detroit, Los Angeles, Norfolk, for example have more than once, asked 
ny professional opinion about new products or strategies. They were con- 
sidering. This need not be expensive - it all depends on what amount of 
work is required. In some cases, organizations might be called on. The 
Center for Applied Linguistics, for example, provides such consulting service 
to many school systems, especially regarding bilingual education and general 
language arts. There are a dozen or so really good and conscientous critic- 
consultants in the reading and language arts area. There must be similar 
people available in other areas as well. They have the advantage of object- 
ivity (to your system',,.at least) and distance. Your own staff may often 
be too close to the situation for such objectivity. 

Any list of those who can assist the administration in making decisions 
relating to innovative products should include his oxm staff, the producer, 
consultants and the clients themselves. Community input at an early stage 
tends to avoid negative community reaction at a later date. 



6. What Areas Are Innovation - Susceptible? Know the difference . 

It has been the case, at least in recent years, that certain areas 
of education are more apt to be "innovated upon" than others. Reading, 
educational management and teaching methods are the clearest cases in point. 
These areas share a number of characteristics which make them highly 
vulnerable to innovation. For one thing, they are all highly visible to 
the general public and this public has not been timid about expre rising 
dissatisfaction with what the schools are d6ing. The areas in wh^ch 
innovations are susceptible also tend to have potential for protit in a 
commercial sense. In contrast other content areas tend to avoid innovative 
intrustions, largely becaase they are not terribly profitable or because 
the public demands that their traditional approaches or content not be 
tampered with. Such areas include the subjects, history, language, economics, 
religion and literature. These areas do not stress skills to a large extent 
and they have the public auro of a tradition which ought Hot to be chai*ged. 
Evidence of this resistance to change can be seen in the areas of grammar 
study. Our knowledge base for grammar study has undergone at least 
revolutions in the past thirty or forty years, neither of which has had 
any significant influence on the way schools teach grammar. Apparently the 
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field of English is not highly susceptible to innovation* On the other 
hand, it is an area of high visibility (we have almost monthly national 
laments for how bad our children write). 

At least one major publisher was very conscious of this traditional 
aspect of the language arts when it developed and produced its comprehensive 
English program. Its marketing people wanted very much to label the program 
revolutionary in terms of linguistics and innovative in terms of methodology* 
The editorial board, however, wisely resisted and requirdd that the new 
linguistic focus be downplayed as a selling device, largely out of knowledge 
that such information was out-of-place in a traditionally perceived market. 
People tend to not want njew things done to their children's writing or 
grammar. They do want good things done. In such cases, the focus on 
newness or innovation was even bad marketing. 

In terms of principles for the wary administrator to apply, it would 
seem that one might urge to beware of innovations in innovation-resistant 
fields. Either the innovator is unaware of the distinctions between 
favored and unfavored public dispositions (cause enough, perhaps, to doubt 
the innovator's perceptions) or the ultimate public judgment of the 
innovation will tend not to be favorable simply because it teads on hallowed 
ground. On the other hand, administrators should be aware that in apparently 
innovation-permissible areas such as teaching methods, classroom management 
and reading, any innovation should be seen in terms of the environment in 
which it operates. Innovations have not been successfully evaluated because 
of failure to see all of the implications involved. There is an ecology of 
education into which the insertion of a new product or strategy cannot be 
seen in isolation. What was the effect of performance contracts on teacher 
or student morale, for example? 

poncluston ! 

What has been presented is a set of suggestions or guideline? £or new 
products, programs or gimmicks which are sometime, dignified by tvfta 
educational innovations. I have suggested, in brief six major cu^.oideraticns: 

!• Know why you are innovating before you innovate. Is it out of des- 
peration or out of Jealousy or fear of being left behind? 

2. Know how to avoid over reacting simply to a nice looking product. 
Art departments are doing fantastically good work these days and your 
natural love of childhood can actually distort your perception of progr.nm. 

3. Know how to be objective about the people part of innovation* Good 
people do not necessarily imply good innovations. 
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4« Carefully examine the difference, or potential difference, between 
the claim of the innovation and the potential for reality. Past claims 
of success can be objectively viewed. 

5t Know the theory or knowledge base which underlies the innovation. 

6. Kaow the difference between innovation<-suscept:ible areas and 
innovation resistant ones. Your choice of success in the later is slim, 
no matter how good the product may be. 

In addition to these guidelines, let me also suggest that the innovation 
not be Judged solely on the basis of the sales person's presentation. Text- 
book selection for the state of Texas rests on such a performance. I once 
observed the process and found that it ludicrously represents the product 
under inspect ion« Nonetheless, the representative of the innovation can 
be global rather than narrow, the representatives have (or should have) 
answers to the specific questions. Ask how and why rather than what . 
In addition ask process questions rather than product ones. Ask the repre- 
sentative what he would do under such circumstances. Ask for definition 
of terms. I once learned that an innovation peddler thought individualization 
referred to learning rather than to teaching. When I asked him how anybody 
could learn in any other way besides individually, he was non«*plussed« If 
the salesperson cannot answer these questions, do not be surprised but expect 
him to get them for you. 

Having gone through these guidelines and suggestions I must now make 
a confession. This confession is that if I were an administrator, the single 
most important thing I could do for educational practice in my district would 
be to insure that my teachers were knowledgeable. As a person who develops 
and helps write teachers manuals and educational programs of several types 
I must admit that materials, techniques, methods, strategies and other 
gimmicks are less important than what the teacher knows. The greatest thing 
we can offer our children is teachers who know how to use these giii,1ellnes. 
Perhaps the time will come when our teacher preparation inctitut:^ will do 
this well. Perhaps the time will come when administrators leart dis- 
tinguish and select only competent and humane teachers. Perhap"-" t\ie day 
will arrive when our priorities move from tricks and fads to scJic knowledge. 
But until such a time arrives I recommend, as a minimum effort, that we 
apply these guidelines suggested here today* 



